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ABSTRACT 



A think- aloud protocol study examined changes in real-time 
cognitive, metacognitive , and motivation processing that may have resulted 
from reading-strategy training. Eighth-grade students participated in 4 weeks 
of reading- strategy training during social studies class. Students used their 
own social studies text during the training. Fourteen participants performed 
think- aloud protocols and comprehension tasks prior to, and following 
strategy training. Significant increases in the proportion of participants’ 
use of one summarization strategy, and in overall use of a composite 
categorization of comprehension strategies were found. No other processing 
changes were identified. Ability to use trained strategies, and comprehension 
performance increased significantly. Significant findings were attributed to 
changes among a few individual readers, and thus, the impact of the training 
may be characterized as limited. Future research might examine real-time and 
performance effects using a variety of training methods, as well as include 
data related to real-time processing during task performance. (Contains 29 
references and 9 tables of data. Appendixes present directions for 
think- aloud training, a think- aloud protocol scoring scheme, examples of 
segmented and scored protocols, and a teacher’s description of training 
procedures . ) (Author/RS) 
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This think-aloud protocol research examined changes in real-time cognitive, metacognitive, and motivation 
processing that may have resulted from reading-strategy training. Eighth-grade students participated in 4 
weeks of reading-strategy training during social studies class. Students used their own social studies text 
during the training. Fourteen participants performed think-aloud protocols and comprehension task prior to, 
and following strategy training. Significant increases in the proportion of participants’ use of one 
summarization strategy, and in overall use of a composite categorization of comprehension strategies were 
found. No other processing changes were identified. Ability to use trained strategies, and comprehension 
performance increased significantly. Discussion focuses on the effectiveness of the strategy training, and 
possible reasons for the limited relationship between think-aloud data and performance data. 
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Examining the real-time effects of reading-strategy training 



Active processing during readine . Readers play an active role in bringing about their own learning 
and comprehension of text (Brown, Campione, & Day, 1981; Chan, Burtis, Scardamalia, & Bereiter, 1992; 
Johnston & Afflerbach, 1985; Kintsch & van Dijk, 1978; Palincsar & Brown, 1984; Paris, Cross, & 
Lipson, 1984). Readers who engage in activity for the purpose of fostering comprehension and learning 
have been described as strategic readers (Paris, Lipson, & Wixson, 1983; Paris, Wasik, & Turner, 1991). 
Strategic readers coordinate metacognitive, cognitive and motivational factors when selecting, using and 
monitoring learning strategies. 



Learning strategies have been defined in a variety of ways - usually in a manner reflective of the 
researcher’s theoretical predilection. Most definitions point to a complex relationship among learners’ 
strategy-use, processing goals, cognitive, and metacognitive processes. Weinstein and Mayer (1986) defined 
strategies as "behaviors and thoughts that a learner engages in during learning and that are intended to 
influence the learner's encoding processes" (p. 315). Pressley, Borkowski, and Schneider (1987) suggested 
that strategies are "operations over and above the processes that are a natural consequence of doing a task" 

(p. 90). These processes have a purpose, and are potentially conscious and potentially controllable. Dole, 
Duffy , Roehler, and Pearson (1991) described comprehension strategies as "conscious, instantiated, and 
flexible plans readers apply and adapt to a variety of texts and tasks" (p. 242). Paris et al. (1991) defined 
strategies as "conscious and deliberate actions selected to achieve particular goals" (p. 61 1), and that require 
both effort and choice of activity. 

More recently, Pressley and Afflerbach (1995) proposed a model of the “constructively responsive 
reader” (pp. 83-84). Like previous descriptions of readers’ strategy-use, this model portrays the reader as 
integrating cognitive, metacognitive, and motivational factors. Unlike previous work, however, Pressley 
and Afflerbach take a more integrative approach and describe how these factors are operating within 
categories of expert-reading activities (e.g., identifying and learning text content, monitoring, and 
evaluating) rather than as isolated processes. 

Effect? of reading-strategy training . A considerable body of research suggests that use of learning 
strategies is positively related to academic performance (e.g., Pintrich & DeGroot, 1990; Zimmerman & 
Martinez-Pons, 1990), and that training students to use learning strategies is effective at enhancing academic 
performance (e.g., Haller, Child, & Walberg, 1988; Hattie, Biggs, & Purdie, 1996). Training readers to use 
comprehension strategies and to self-regulate their strategy use enhances their comprehension performance 
(e.g., Baumann, Seifert-Kessell, & Jones, 1992; Bereiter & Bird, 1985; Palincsar & Brown, 1984 - Paris et 
al., 1984; Schunk & Rice, 1991, 1993). 



At present, most explanations of training effectiveness attribute enhanced reading performance to 
changes in underlying processes (see discussions in Bereiter & Bird, 1985; Paris et al., 1984; Schunk & 
Rice, 1991, 1993) which, in turn, result in increased use of learning strategies. Evidence supporting the 
correspondence between changes in comprehension performance and the underlying processes, however, has 
not been forthcoming. 

Examining real-time effects of training . In a recent review of reciprocal teaching, Rosenshine and 
Meister (1994) pointed to the need for investigations into changes in readers’ mental processes associated 
with strategy training. Rosenshine and Meister noted that virtually no research has been conducted that 
could provide information about how reciprocal teaching (and more broadly, cognitive strategy training) 
influences students* cognitive processes. 



One exception cited in Rosenshine and Meister (1994) was Bereiter and Bird (1985), who showed 
that strategy training did enhance readers’ real-time use of trained strategies. Bereiter and Bird, used think- 
aloud protocols to identify expert comprehension strategies and to "assay the prospects for use of thinking 
aloud as an instructional vehicle (p. 140) to teach middle-school readers to use expert comprehension 
strategies. Four comprehension strategies were identified from expert-readers' think-aloud protocols, and 
middle-school readers were then trained to use these expert-comprehension strategies while performing a 
think-aloud protocol. At pretest, readers were given approximately 40 min of instruction on how to 
perform a think-aloud protocol and how to use 15 different comprehension strategies. Participants then 
individually performed a think-aloud protocol while reading six passages from a standardized comprehension 
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test. Pretest think-alouds were analyzed to determine the extent to which middle-school readers used the 
expert-comprehension strategies. 

Following the pretest, participants were assigned to one of three treatment groups which varied the 
availability of modeling and explanation of how to identify and use the expert-comprehension strategies in 
the context of performing a think-aloud protocol. Participants received nine 40-minute training session 
over a three-week period. Posttests consisted of a silent-reading comprehension test and a think-aloud 
protocol while reading six different passage from the same standardized reading test. 

Results from Bereiter and Bird (1985) indicated that during the think-aloud protocol, students who 
were given modeling and explanation for performing think-aloud protocols and use of the four expert- 
comprehension strategies exhibited three of the expert strategies significantly more than readers in any other 
treatment group and they performed significantly better on the silent-reading comprehension test. 
Correlational analyses, however, failed to identify any relationship between strategy use and comprehension 
performance. 

Bereiter and Bird (1985) demonstrated an important relationship between comprehension- 
monitoring training and real-time strategy-use. Their research was limited, however, in that only readers’ 
use of cognitive monitoring strategies were examined, and their research question focused on the feasibility 
of using think-aloud protocol training as an instructional method. Examinations of processes other than 
comprehension-monitoring and the influence of think-aloud training are needed in order to address the issue 
raised by Rosenshine and Meister (1994). 

Focus of the present research The present study is intended to extend current understanding of 
strategy-training research by identifying the effects of strategy training on the cognitive, metacognitive, and 
motivational processes readers’ actually engage in during reading. In this study, eighth-grade readers 
participated in strategy training using the reciprocal teaching method as described by Palincsar and Brown 
(1984). Prior to and following strategy training, participants performed a think-aloud protocol while 
reading from their regular social studies textbook. After reading, participants completed a teacher- 
constructed comprehension test. Training consisted of 4 weeks of reading strategy training using the 
reciprocal teaching method. Training was conducted in the context of students’ social studies class, and 
included weekly measures of students’ progress in learning to use the 4 trained strategies. 

For this study, there were three hypotheses. First, strategy training will increase students’ ability 
to use trained strategies. Second, strategy training will increase readers’ real-time use of strategies while 
engaged in the reading task. Third, reading strategy training will enhance students’ comprehension 
performance. 

METHODS 

Participants 

Participants were 14 students (Girls = 6; Boys = 8) from an eighth-grade class of 24 students, at a 
middle-size parochial school in Austin, Texas enrolled during the 1994-1995 school year. Participants 
represented different ethnic and racial groups: 7 Hispanic Americans (50%), 6 Anglo Americans (43%), and 
1 Asian American (7%). For all participants, English was their primary language. Participants’ average 
age at the time of the initial data collection was approximately 14 years old (Girls = 13 years, 1 1 months; 
Boys = 14 years, 1 month). 

Participants’ reading comprehension scores on the Comprehensive Test of Basic Skills (CTBS) 
(1989) from seventh-grade ranged from 829 (12th grade equivalent) to 713 (fifth-grade equivalent). Average 
scale score for all participants was 762 (9.2 grade equivalent). 

Materials 

Reading passages for both pre- and post-training were taken from America: The Glorious 
Republic (Graff, 1991). This social studies textbook was used as part of the regular social studies 
curriculum throughout the school year. Training passages were selected based upon the sequence of 
classroom instruction. The passages selected for reading at pre- and post-training were those that 
immediately followed the previous day's instructional assignment. 

Measures 

Think-alo ud protocols . Measures of participants' real-time processes during reading were obtained 
using a think-aloud methodology (Ericsson & Simon, 1984; Pressley & Afflerbach, 1995). Participants 
were given instruction about how to perform a during-reading think-aloud protocol (see Appendix A for 
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experimenter’s directions) and asked to practice performing the think-aloud. Practice was terminated when 
the experimenter judged that participants were adequately verbalizing their thoughts during reading. The 
average practice-time was approximately 16 minutes (minimum = 5 min; maximum = 35 min). 

Participants were then asked to perform the think-aloud task while reading from the target passage in their 
social studies textbook. Participants' verbalizations were audio taped and a companion video tape was used 
to record participants' behavior while they were engaged in the reading task. 

Readers’ verbalizations were coded using contexutalization notes (i.e., notes about readers’ behavior 
prior to, during, and after each statement) from the companion video and a scoring scheme modified from 
previous protocol research (Shell, 1991). This scheme was used to categorize statements as pertaining to 
text processing (5 subsets), prior knowledge (5 subsets), metacognition (5 subsets), or motivation (4 
subsets) (see Appendix B for the scoring scheme). Within the text-processing and prior-knowledge 
categories, additional subcategories of processes were included in order to reflect the specific strategies 
included in Palincsar and Brown (1984) . These subcategories included verbalizations relating to 
summarization, clarification and predicting. Protocol statements were coded independently by three 
researchers. Following independent coding, any discrepancies were resolved by discussion until agreement 
was reached (see Appendix C for examples of scored protocols, pre- and posttest). 

Shell’s (1991) scoring scheme was very similar to the model proposed by Pressley and Afflerbach 
(1995). Pressley and Afflerbach proposed three broad categories for describing readers’ real-time processes: 
meaning-making, monitoring, and evaluation. Shell utilized a similar approach to classifying statements 
(e.g., meaning-making ’ corresponded with text-processing and prior-knowledge processing; “monitoring” 
corresponds with metacognitive processing). One difference in the schemes, however, was that Shell 
included “evaluation” processes as part of metacognitive processing, rather than as a separate category. This 
difference seems to reflect differences in emphasis only. Pressley and Afflerbach (1995) created qualitative 
categories to reflect processes identified in previous research. Shell created categories that more closely 
reflected current understanding of cognitive theories. 

Strategy-use manipulation check. In order to assess progress in learning to use each of the trained 
strategies, participants completed a manipulation check on the Friday of each week during the training phase 
of the study. The manipulation check consisted of a brief reading passage (2 to 6 paragraphs) selected from 
the current chapter of instruction in the social studies textbook, and a questionnaire on which participants 
wrote (a) a summary sentence of each paragraph, (b) a main idea question, and (c) a question or statement 
requesting clarification of any part of the paragraph that was unclear or which was lacking in the text. 
Participants' responses to the manipulation check were scored using the same criteria that the teacher used to 
respond to participants' strategy use during the group training throughout the week. For each of the four 
manipulation checks, participants were given a score for summarization, main-idea-questioning, and 
clarification strategy based upon a ratio of correct responses to possible responses. 

C omprehension performance . Comprehension skill was measured using a teacher-constructed, 
short-answer test based on the section review questions at the end of the experimental text passage. 
Comprehension measures were scored by the classroom teacher-experimenter. 

Procedure 



Three types of data were collected in 3 phases over the course of a six-week period: (a) think-aloud 
protocols of students reading from their social studies textbook, (b) strategy-use manipulation checks, and 
(c) comprehension performance. 



Pre- training . In the first phase, participants performed a think-aloud protocol while reading an 
assigned section from their social studies textbook. Each students performed the think-aloud protocol in a 
room separate from the classroom, and in the presence of a graduate-student experimenter. After completing 
the reading, participants completed a comprehension quiz over the text passage they had just previously 
read. In order to enhance the authenticity of the experimental task, participants were informed that their 
performance on the quiz would be counted as a class grade. 

Training . The second phase of the study was begun the following week after all participants had 
completed the pre-training portion of the study. In this phase of the study, which lasted 4 weeks, students 
participated in strategy instruction utilizing the reciprocal teaching method (Palincsar & Brown, 1984). The 
training took place during students’ social studies class, and students were taught by their regular classroom 
teacher. Students learned to use the strategies while reading from their social studies textbook On the last 
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day of each week, participants completed the strategy-use manipulation check (see Appendix D for a fuller 
description of the training procedure). 

East-training. Approximately one week following the end of the training, the third phase of the 
study was begun. During the post-training phase, participants performed a think-aloud protocol while 
reading from their social studies textbook and completed a comprehension quiz over the text passage which 
they had just previously read. Participants were again informed that their performance on the quiz would be 
counted as a class grade. 

RESULTS 

Overview of Data Analysis Procedures 

S tatistical power and alpha inflation . In order to increase the power of statistical tests for detecting 
possible training effects, we adopted an alpha level of . 10 for this study. There were two reasons for this 
decision. First, the study was exploratory in nature. The exact nature and magnitude of changes that could 
be expected in this study have not been established by previous research. Second, the sample size was 
relatively small ON = 14). With a sample-size this small, traditional alpha settings may have required the 
magnitude of change attributable to training to be unrealistically large. 

I bink-aloyd prQtocols . Every effort was made to train participants to perform the think-aloud 
protocol. Still, participants differed substantially from each other in the total number of statements made 
while performing the think-aloud protocols. Given this situation, analyses of think-aloud differences using 
frequencies of statements would not have provided an accurate description of changes that might have 
occurred as a result of strategy training. For this reason, we conducted statistical analyses on proportions of 
a particular statement category (based on the ratio of the number of statements in a category to the total 
number of statements made during the think-aloud). 

We conducted the analysis of differences between proportions of pre-training and post-training 
think-aloud statement categories in two steps. First, we calculated an overall effect size for each category 
and subcategory of think-aloud statements. Second, if the effect size was .33 or greater, we performed a 
dependent-measure t-test. We decided that an effect size of .33 (equivalent to one-third of a standard 
deviation) could reasonably be assumed to reflect a real change in processing. 

Analyses of Training Effects 

Thiqk-frloud protocols . Table 1 shows a summary of the proportions of think-aloud statements 
(both pre- and posttest) coded into each category. Table 2 shows the amount of time readers spent 
practicing the think-aloud procedure, and performing the pretest think-aloud. This indices was included in 
order to provide a clearer picture of the context in which the think-aloud protocols were performed (see 
Pressley & Afflerbach, 1995, pp. 8-13). 

Table 3 shows the proportions and effect sizes of real-time processing for text-processing 
statements and five subsets of this primary category. There was a significant change (effect size greater than 
.33) found for participants’ use of deep-processing summarization strategies, t (13) = 2. 12, g < .02. This 
change was due to increases by 5 readers (35%), while 8 readers (56%) did not change in their use of this 
strategy. Table 4 shows the proportion and effect sizes of real-time processing for prior knowledge 
processing and five subsets of this primary category. There was a decrease in the use of prior-knowledge 
summarization, t (13) = -1.31, but this change was not significant. No other prior-knowledge processing 
category had effect sizes large enough to warrant further testing. Table 5 shows the proportion of total 
metacognitive statements made during the think-aloud protocol, and proportions of five subsets of this 
primary category. There was an increase among 8 readers (57%) in the statements of metacognitive 
evaluation; and a decrease in personal-monitoring statements among 5 readers (36%). Neither of these 
changes, however, were statistically significant at the .10 level. 

Table 6 shows the proportion of total motivational statements made during the think-aloud 
protocol and proportions of four subsets of this primary category. There were no increases in statements 
related to motivational processes following training. The findings presented in Tables 3, 4, 5, and 6 
indicate strategy training may have had a limited effect on readers’ use of text-processing strategies (i.e., 

deep-processing summarization), but other processes typically associated with strategy-use remained 
unchanged. 

S trategy-use manipulation checks . Table 7 contains the means of participants' performance scores 
for strate gy use on the four training-phase manipulation checks. Due to absences on days when the 
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manipulation checks were administered, the complete sample for this analysis was 1 1 participants. Those 
participants who missed one or more of the manipulation checks, however, conformed to the pattern 
described below. From a repeated measures ANOVA, significant linear increases were found in participants' 
ability, over time, to use both the summarization strategy, E (3, 27) = 24.75, g = .000, and the main-idea 
questioning strategy, E (3, 27) = 5. 18, g = .006. No change was found for participants' ability to use the 
clarification strategy. The findings presented in Table 5 indicate that participants' ability to use two of the 
trained strategies did significantly increase over the course of training. 

Comprehension performance . Table 8 contains the means of participants' performance scores on 
the post-reading comprehension quiz. A significant difference was found for participants' comprehension 
from pre-training to post-training, t (13) = 4.62, g = .01. These findings indicate that reading 
comprehension increased significantly from pre-training to post-training. The increases in reading 
comprehension performance, however, represents significant, but modest gains in comprehension 
performance. The scores for post-training comprehension performance still did not reach a level considered 
to be acceptable in typical classroom setting. 

Post Hoc Analysis Of Comp o site Processing Variables 

For a broader analysis of the effects of strategy training on readers' real-time processing during 
reading, we formed composite variables of each of the main scoring categories. The primary reason behind 
the formation of these composite variables was that the proportions of statements in any of the individual 
categories was relatively low. By combining categories according to similarity of processing, the 
proportion of similar processing statements would be increased and, therefore, training effects might be 
more readily detected. 

We formed five composite variables: (a) total-metacognition, consisting of statements categorized 
as related to metacognitive processes, excluding personal and environmental monitoring, (i.e., monitoring, 
nietacognitive control, metacognitive evaluation); (b) total-summarization, consisting of all summarization 
statements in the text-processing and prior knowledge categories; (c) total-clarification, consisting of all 
clarification statements in the text-processing and prior knowledge categories; (d) total trained-strategies, 
consisting of a combination of all summarization and clarification statements in the text-processing and 
prior knowledge categories; (e) total-comprehension strategies, consisting of the total trained-strategies, 
with the addition of deep-processing inference and prior-knowledge inference categories. 

We formed composite variables for summarization and clarification because whether or not prior 
knowledge is involved, they represented similar types of processing in the two categories. We formed a 
composite variable of total trained-strategies to provide a better overall picture of the effects of reading- 
strategy training on readers' real-time comprehension processes. We formed a total-comprehension 
strategies variable to provide an overall picture of the effects of training on processes described as fostering 
comprehension. 

Table 9 shows the proportion of composite variable statements made during the think-aloud 
protocol. There were increases (effect size greater than .33) in three composite variable categories. The 
increase in total-comprehension strategy use was significant (p = .07). The increases in total- 
summarization and total-trained strategies were not significant. 

The results presented in Tables 9 indicated that there may have been a limited training effect on 
participants composite comprehension strategy use during reading. As was the case with previous 
analyses, significant differences detected on dependent measures t tests were driven by large increases among 
a few individuals rather than by changes across the entire group. In the case of total-comprehension 
strategies, however, 7 participants (50%) did show an increase in strategy use. Nevertheless, these findings 

call into question the extent to which training may have led to changes in students' real-time processing 
during reading. 



DISCUSSION 

Summary Of Results 

The purpose of this research was to investigate the effects of strategy training on middle-school 
O students ’ real-time processes during reading. We hypothesized that training would bring about positive 
£ RJC ° han S es ‘ n rea l-hme processing, strategy skill, and comprehension performance; and specifically that 
increases in comprehension performance would be associated with positive changes in cognitive, 
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metacognitive and motivation processing. Analysis of think-aloud protocols indicated some support for 
these hypotheses. For example, readers exhibited a statistically significant increase in the proportion of 
text-processing summarization statements (see Table 3). Post hoc analyses of composite statement- 
categories indicated increases in use of comprehension strategies (see Table 9). Participants’ ability to use 
two of the three trained strategies increased significantly over the course of the training (see Table 7). 
Performance on teacher-made comprehension measures indicated positive gains attributable to strategy 
training. 

Although the unit of analysis for this study was the whole-group, examination of changes among 
individuals revealed an interesting situation. Although summarization processing increased significantly, 
the number of individual readers who increased their text-processing summarization statements was 5 
participants (35%); and the number who increased their composite comprehension statements was 7 
participants (50%). Thus, even with significant changes in some real-time processing categories, the extent 
of the changes seemed to be limited to changes in processing associated with less than a majority of the 
participants. This outcome is striking in the sense that training might be assumed to influence all students 
in some manner. In the present case, significant findings can be attributed to changes among a few 
individual readers, and thus, the impact of the training may be characterized as limited. 



Examining Reasons for Limited F.ffects 

Results from this study provide evidence that some changes in real-time processing may have 
occurred as a result of the training intervention. The limited nature of the changes, however, raised two 
important questions related to the (a) effectiveness of the training implementation specifically, and (b) 
relationship between think-aloud and performance data in general. 

Implementation of training . Reciprocal teaching of reading strategies was the training method 
selected for this study. We were not attempting to evaluate the effectiveness of reciprocal teaching methods, 
however, previous criticisms of this method may offer insight into the outcome of the present study. 

The adequacy of this method for enhancing comprehension, and the manner in which it was 
implemented in the present intervention may have influenced the outcome of the present study. Pressley 
and McCormick (1995, p. 243) suggest that the reciprocal-teaching method may not' be as effective as other 
methods for fostering changes in higher-order comprehension processing. Specifically, Pressley and 
McCormick concluded that lack of teacher-support during readers’ group exchanges may be responsible for 
students exhibiting lower-level comprehension activities (e.g., literal questioning and lack of clarification 
activity). 

In the present study, teacher-direction was not faded to the degree reported by other reciprocal 
training research (see Appendix D for teacher’s description of training). The teacher continued to meet with 
each group and direct students’ task involvement. In this case, the teacher’s continued involvement in the 
group reading activity may have ensured a higher-level of comprehension processing. Teacher involvement, 
however, may also have hinder students’ independent use of the reading strategies. Weekly manipulation 
checks indicated that students did know how to use the strategies (Table 7), but they may have lacked the 
conditional knowledge (i.e., when to use a particular strategy, and why it is important to use the strategies) 
necessary to use the strategies independently. A lower degree of teacher involvement may have helped 
students acquire this information. The limited nature of changes in students’ real-time processing, therefore, 
may be a result of the manner in which the training was implemented. 

The relationship between rea l-time processes and comprehension performance . Judged by 
comprehension performance alone, the strategy-training intervention seemed to have had a positive impact 
on these middle-school readers (Table 8). Changes in comprehension performance have traditionally been 
linked to changes in underlying processes (i.e., cognitive, metacognitive, and motivational processes). For 
example, Paris et al. (1984) stated, 

We can infer that they [the trained students] used instructed strategies, such as using 
surrounding context to supply unknown words and monitoring consistency and 
sensibility of text to succeed on these tasks that require comprehension strategies (p. 

1248). ... We can infer from their increased performance on strategic tasks [i.e., 
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cloze test and error detection task] that they also learned how to evaluate, plan and 
regulate their reading (p. 1250). 

The present analyses, however, suggest that this inference may not be entirely accurate. 

The lack of corroboration between real-time processes and measures of the comprehension 
performance was particularly striking in this study. Previously, however, researchers have reported similar 
findings. For example, Bereiter and Bird (1985) showed that training readers to think-aloud during reading 
and to use expert monitoring strategies increased their use of comprehension-monitoring strategies at 
posttest. No relationship was found between use of comprehension strategies and comprehension 
performance. Wade, Trathen, and Schraw (1990) found Qualitative differences in the manner of text 
studying, but failed to find any significant differences between strategic activity and performance on a recall 
measure. Baumann, Seifert- Kessel, and Jones (1992) trained readers to use either a think-aloud method or 
directed-thinking activities. Although qualitative evidence suggested that think-aloud readers “demonstrated 
a greater depth and breadth of comprehension monitoring abilities” (pp. 143-144), no performance 
differences were found. Compared with the results of these previous studies, the outcome of the present 
research again raises the question of the relationship between real-time processes and performance (Bereiter 
& Bird, 1985). 

Examining possible sources of d iscrepancy . The lack of corroboration between verbal-report and 
performance data has raised some concerns about the accuracy of think-aloud data, and more generally the 
relationship between cognitive/metacognitive processes and performance (Bereiter & Bird, 1985). 

Proponents of the think-aloud methodology point to several procedural precautions that may ensure the 
reliability and validity of this data source (Pressley & Afflerbach, 1995). A closer examination of the data 
sources, however, reveals at least two potential reasons why closer correspondence has not been established. 

First, students’ ability to perform a think-aloud protocol may be more closely related to factors 
such as verbal ability rather than to processing ability, or related to degree of automaticity of processes 
rather than presence or absence of processing. Differences in verbal ability and processing automaticity 
may, therefore, mask differences in real-time processing. In the present study, we attempted to compensate 
for this difference by calculating proportions of statements that reflected a processing category. Second, 
there may be a fundamental difference between the constructive task of reading text and the reconstructive 
task of responding to comprehension questions. The think-aloud methodology is intended to provide 
statements reflective of general categories of processing, whereas the performance measure is intended to 
reflect acquisition of specific information. Therefore, think-aloud performance during reading and recall 
processes during performance may not be closely related. One solution may be to examine more closely the 
* con tent” of think-aloud statements, in addition to describing “process”; or to request subjects to continue 
the think-aloud protocol during completion of performance tasks. In this manner, researchers might gain a 
better insight into how content is retrieved from memory and the relationship between reading processes and 
memory processes during task performance. 

Implications an d Directions for Research and Practice 

At present, the relationship between real-time processes and task performance seems to be the most 
important theoretical question raised by the present research. Understanding how think-aloud and 
performance data are related is essential for gaining a better understanding of the impact that strategy 
training might have on students’ real-time strategic processes. In the future, researchers might examine 
real-time and performance effects using a variety of training methods, as well as include data related to real- 
time processing during task performance. 

Attention to learning-strategy training seems to be waning among the research community (much 
like metacognition did during the late 1980’s) at a time when interest is increasing among educational 
practitioners. Results from this, and other studies suggest that classroom teachers can be effective agents of 
strategy instruction (e.g., Moely et al, 1992; Palincsar & Brown, 1984; Pressley & Woloshyn, 1995). In 
the future, researchers and practitioners together might utilize think-aloud methodology to examine real-time 
changes in processing, as well as identify teacher-characteristics and actions that are most closely identified 
with students’ self-directed use of learning strategies. 
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Table 1 

Mean Proportion of Think-Aloud Statements 






During-reading 
Strategic Process* 


Pre-training 


Post-training 


Total Text - 
Processing 

M 


.11 


.17 


Total Prior- 
Knowledge 
Processing 

M 


.30 


.31 


Total 

Metacognitive 

Processing 

M 


.35 


.35 


Total 

Motivation 

Statements 

M 


.13 


.13 


Unscorable 

M 


.11 


.06 



Note. Columns may not add to 1 .00 due to rounding. 



Table 2 

Mean Practice Time and Reading Time at Pretest 



Variable 


M 


SD 


Minimum 


Maximum 


Practice time 
Reading time 


17.29 

16.50 


9.50 

6.31 


5 min 
11 min 


35 min 

36 min 



Note, n = 14. 
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Table 3 

Mean Proportions of During-R e ading Text Processing 
Strategic 

Processes Pre-training Post-training 



Effect Size 



I value 



Total Text - 
Processing 
M 
SD 



Primary Scoring Category 



•11 .17 

•12 .21 



.36 



.95 



Surface 

Processing 

M 


.07 


Scoring Sub-Categories 
.06 


SD 


.11 


.07 


Deep-Processing 

Inference 

M 


.01 


.02 


SD 


.01 


.06 


Deep-Processing 

Summarization 

M 


.00 


.04 


SD 


.01 


.06 


Deep-Processing 

Clarification 

M 


.02 


.05 


SD 


.03 


.11 


Deep-Processing 

Predicting 

M 


.00 


.00 


SD 


.00 


.00 



Hole* Tests not performed (na) on variables with effect sizes less than .33. 
*£ <.10 



na 



na 



2 . 12 * 



1.03 



na 



O 

ERIC 
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Table 4 

M ean Proportions Qf Puring-Reading Prior- K nowledge Processing 

Strategic Processes Pre-training Post-training Effect Size £ value 



Primary Scoring Category 

Total Prior-Knowledge 
Processing 

M .30 .31 

SD .19 24 



.05 



na 



Prior-Knowledge 

Inference 

M 

SD 

Prior-Knowledge 

Summarization 

M 

SD 

Prior-Knowledge 

Clarification 

M 

SD 

Prior-Knowledge 

Prediction 

M 

SD 

Prior-Knowledge 

Personal. 

M 

SD 



Scoring Sub-Categories 



13 .14 .10 

•10 .15 



•02 .01 .39 

•02 .03 



•03 .03 .00 

•05 .05 



•00 .00 .00 

• 00 .00 



12 -12 .08 

• 08 .12 



Nfile, Tests not performed (na) on variables with effect sizes less than 33 

*£< .10 



na 



-1.31 



na 



na 



na 
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Table 5 










Mean Proportions of Purine-Reading Metacoenitive Processing 
















Strategic 

Processes 


Pre-training 


Post-training 


Effect Size 


I value 






Primary Scoring Category 






Total 

Metacognition 

M 


.35 


.35 


.01 


na 


SD 


.22 


.33 










Scoring Sub-Categories 






Monitoring- 

Comprehension 

M 


.15 


.14 


.04 


na 


SD 


.18 


.19 






Monitoring- 

Environment 

M 


.07 


.06 


.15 


na 


SD 


.06 


.07 






Monitoring- 

Personal 

M 


.02 


.01 


.57 


-1.78 


SD 


.04 


.02 






Metacognitive 

Control 

M 


.03 


.03 


.11 


na 


SD 


.04 


.05 






Metacognitive 

Evaluation 

M 


.08 


.13 


.39 


1.08 


SD 


.07 


.16 







Note. Tests not performed (na) on variables with effect sizes less than .33. 
*& < .10 
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Table 6 

Mean Proportion of Purine-Reading Motivational Statements 
















Motivation 

Processes 


Pre-training 


. Post-training 


Effect Size 


I value 


Total Motivation 
Statements 
M 
SD 


.13 

.10 


Primary Scoring Category 

.13 

.20 


.03 


na 



Motivation Sub-Categories 



Interest 



M 


.06 


.04 


.30 


na 


SD 


.07 


.06 






Causal Attributions 

M 


.01 


.01 


.02 


na 


SD 


.03 


.05 






Self-Efficacy 

M 


.00 


.00 


.00 


na 


SD 


.00 


.00 






General Motivation 

M 


.05 


.05 


.03 


na 


SD 


.10 


.14 







Note. Tests not performed (na) on variables with effect sizes less than .33. 

*£ < .10 
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Table 7 

Me ans Of Strategy-Use Performance on Manipulation Checks 



Manipulation Check 

Strategy Time 1 Time 2 Time 3 Time 4 p value 



Summarization 



M 


.17 


.44 


.80 


.90 


SD 


.20 


.19 


.30 


.16 


Main-Idea Question 


M 


.35 


.54 


.42 


.76 


SD 


.29 


.22 


.42 


.27 


Clarification 


M 


.50 


.58 


.52 


.53 


SD 


.29 


.33 


.40 


.32 



Note. n= 11. 



Table 8 

Means of Comprehe nsion Performance at Pre- and Post-Training 



Variable Pre-Training Post-Training p value 3 



Comprehension 

Performance 

M .34 

SD .18 



.49 

.19 



.01 




Note. a One-tailed significance test. 
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Table 9 










Mean Proportion of Purine-Reading Composite Processing Statements 
















Composite 

Processes 


Pre-training 


Post-training 


Effect Size 


i value 


Total-Metacognition 

M 


.26 


.29 


.11 


na 


SD 


.20 


.33 






Total-Summarization 

M 


.02 


.04 


.33 


1.26 


SD 


.02 


.07 






Total-Clarification 

M 


.05 


.08 


.27 


na 


SD 


.08 


.14 






Total-Trained 

Strategies 

M 


.07 


.13 


.42 


1.07 


SD 


.09 


.18 






Total-Comprehension 

Strategies 

M 


.21 


.29 


.38 


1.51* 


SD 


.16 


.26 







Note. Tests not performed (na) on variables with effect sizes less than .33. 
*£<•10 
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Overview of Procedures 

1 . Introduction and directions for the think-aloud 

2. Practice 

3. Reading task with think-aloud (record with video and audio) 

4. Comprehension quiz 

Introduction and Directions for the Think-Aloud 

A. Make subjects feel at ease with you, the equipment, and the procedures. It is important to remember you are 
working with volunteers, but you are also making a demand on them. 

B. In your introduction, let them know that this is going to be “challenging” to do, so it is really important that 
they listen to directions and ask questions if they do not understand. 

C. Directions: 

1 . Say something like this... 

a. “I want you to say out loud anything that you are thinking while you are reading and trying to understand 
the passage from your history book. 

b. Some things you might be thinking include ... 

1) I’m hungry, tired, bored, etc. 

2) This is interesting, important, fascinating , etc. 

3) You might be summarizing a section after you’ve read. 

(For example, ‘This part was all about how the Homestead Act helped spread small 
farms across the mid-west and kept these states “free states.”) 

c. You might think things like ... 

1) I need to concentrate. 

2) I’m going to look for the main idea. 

3) I’m going to reread that part. 

4) I didn’t understand that part. 

5) I seem to be doing pretty good because I’m concentrating. 

6) Mr. Hamman is a jerk 

2. Say anything you are thinking to yourself, but what I’m most interested in is the stuff you are doing, while 
you are reading, that helps you to understand and remember what you’ve just read. 

It is also really important that you treat this like a class assignment. You are reading this as if you are 
preparing to answer Section Review Questions or to take a quiz (which I’ll give you after you read). 

Practice Session with Think-Aloud 

A. Prepare subjects for the following ... 

B. Heavy prompting during practice (record approximate time each student takes for training) 

1. Subjects should be heavily prompted during this session to familiarize them with the level of think-aloud 
statements we want. 

2. Subjects should be prompted here following silences of more than 10 seconds. 

3. Subjects reading aloud should be stopped and asked (1) if they normally read aloud, and if so continue; (2) to 
say not only what they are reading, but what they are thinking while reading aloud. 

4. During practice, feel free to interrupt, “interrogate” and otherwise direct students activities. You may want 
to ask them about the reading, what they were thinking during reading, what the reading was about, why they 
didn’t say anything aloud, or indicate that their level of statements was satisfactory. 

5. Tell them during the experiment that you will only say, “Remember to say out loud what you are thinking.” 

By this, let them know you do not wish to “strike up” a conversation about the reading, but simply want 
her/him to verbalize. 

Reading Task with Think- Aloud 

A. Show subjects selected passage and where it ends. 

B. Students are not required to read insets to pass the quiz. Don’t say anything unless directly asked. 

C. Prompting readers: If a reader falls silent for more than 20 seconds, prompt them by saying, “Remember to say 
out loud what you are thinking.” Do not ask them directly what they are thinking (like you did during practice). 
Readers should not be making statements for your benefit, but only to verbalize what they are thinking. 

D. If a reader does not verbalize after two prompts ... 

1 . stop them from reading 

2. ask them where they are reading, what it is about, what do they think about it ... 

3. WHY they are not saying anything aloud 

4. You may do this no more than three times. If they persist in NOT TALKING, simply let them finish the 
passage, interview them, give them the quiz and send them on their way. 

E. Any questions about how to read, or marking in the book should be responded to by saying, “Do whatever you 
normally would do to read and prepare for class or a quiz.” 

Comprehension & Recall Quiz 

A. Readers may not use their books to locate answers 
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Metacoeniti ve Categories 



Think-aloud Protocol Scoring Scheme 



1. Monitoring ( MON) - tracking of cognitive activity or resources, such as comprehension, attention, concentration, 

fatigue, strategies, etc. Used for during process tracking only - i.e., the statement occurs in the context of processing something else 
or at the beginning or end of a process. b 

a - Awareness Of Environmental Factors (MON-ENVIRO) - statements relating to distractions inside the experimental 
setting, such as attention given to recording equipment or reference to the novelty of the experimental task: and distractions outside 
the experimental setting such as attention given to noises outside the experimental room, such as talking or laughter from students 
passing by the experimental room. 

b - Awargnegg of Personal Needs (MON-PER) - statements related to distractions within the subject, such as attention 
given to fatigue, allergies, or other bodily functions 



2. Contro l (M-Control) - the aware, deliberate use of a strategy or the conscious redirecting of processing. Generally 
indicated by a change in ongoing processing. Decision making about strategies or approaches to reading. Is also used for 
hypotheses or questions that are used to direct a specific search for an answer (this is indicated by looking for the answer in the text 
and usually by some recognition that the answer was found in a later statement). 

3. Evaluation (M-veal) - statements reflecting evaluation, judgments, critiques of processing or the text. Evaluation is 
NOT used when the judgment, etc. is comprehension or content related (such as a conclusion or agreement with points make in the 
article), it is used for non-content based judgments such as comments about writing style, word use, organization of the article It is 
S i ds ? fo t r K gI °K e , valuanons of strategies or comprehension. This is similar to monitoring, but is used when the statement occurs 

off-line rather than during processing. It issued for more global evaluations such as Tm not getting any of this" or "This approach 
isn t working done during a break in processing. It is also used for recognition that the answer to a hypothesis or question has been 



Text-Processing Categories 



1- - Surface processing (SP) - rehearsal type processing - verbatim statement of text; paraphrase or restatement of what was 

just read. Always done on the immediate (local) text being read; thus, do not use this category for summaries or integration of 
larger text units such as paragraphs. & 



2. Deep processin g 

a * B eep processing-inference (DP-I) - elaborate processing - there are two types. The first is characterized by 
operation on large units of text that go beyond the immediate local text that has just been read. This includes integration or 
summaries of units of text like a paragraph; integration of material from more than one text location; or imposing organization or 
structure. The second type is processing that extends or elaborates text, regardless of whether the text is local or global. The key 
here is that the processing adds" something to what was given in the text. This includes drawing conclusions, inferences, debating 

• f et< T J his cate S 0I 7 is not used if the statement contains knowledge that is not present in or could be 

reasonably be inferred directly from the reading. 

b- S ummarizing ( DP-S) - summarizing larger units of text, such as paragraphs, or sections of the text under a common 
sub-heading. This summarization will typically be more local, but may involve summarization over several paragraphs (e g "So 
he got it, he wanted it, no, he wanted it, he got it, he sold it” ref. Napoleon and the Louisiana territory) 

c. Cl aa fixation (DP-C) - identifying any portion from the text that was unclear to the reader or formulating a question 

"r^ f0rm r n about con ' ent not included in the text, but which is relevant to the topic; seeking an explanation for a 

could do whh'th^MnW 1 £ P[ esented ns fact, but not supported. (e.g„ ”1 wonder what made Spain not even think about what they 
could do with the United States when they turned Louisiana back over to France?”). 3 

d. Predicting (DP-P) - reasoning, based on current text content, about what subsequent text content will include. 



Pnor Knowledge C ategories 



1- Prior Knowl edge Processing 

. a ‘ I nference (PK I) - use of prior knowledge to enhance comprehension, understanding, or remembering of content, 
rimilfr 3 s P eciflc *. inkin 8 of P rior knowledge information with text information. The statement will usually be 

nr^pm n , h f ? und h ? tex ? P rocessin g stra ' e gy ; deep processing category. You should use the PK-rel coding if knowledge not 
inference etc * tr^have^been made statement or ^ 1 s c * ear tbat some knowledge not given in the text is required for the conclusion, 

b. S ummarizing T PK-S) - summarizing larger units of text, such as paragraphs, or sections of the text under a common 

sub-hetiding. This summarization will typically be more global involving information not directly contained in the current text 
passage. 

c - Cl arificat i on (PK-C) - identifying any portion from the text that was unclear to the reader or formulating a question 

ba *f d up ° n outslde ’ or Prior knowledge, that involves information about content not included in the text, but which is relevant to the 
topic, seeking an explanation for a statement in the text that is presented as fact, but not supported. 

d. B rgdicting (PK-P) - reasoning, based on prior knowledge, about what subsequent text content will include. 



2 - Bri or know l edge personal (PK per) - reference to prior knowledge, usually in the form of a personal, episodic experience 

k ^ P ™iiv?nHvr,pHK dge d0CS n0t a P pear t0 be used enhan J ce comprehension, understanding, or remembering of content. This 
IS generally indicated by a statement that contains pnor knowledge with no references to text information. 



a on?v^r^n,.onl y r SC r 0re 35 pn0r kaowledge there must ** some information in the statement that is not contained in the text The 
pwn s for inferences conclusions or questions about content that clearly imply that prior knowledge must be being used 

nI »viiM v (l , e J ' the mfer ence, etc. doesn’t appear to be made able based on the text information alone). If in doubt when 
no explicit pnor knowledge is stated, score as a text processing strategy. 

PI BEST COPY AVAILABLE 



jc 
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Motivation Categories 

1. Interest (MOT-INT) - statements pertaining to interest or lack of interest (that's interesting, this is boring, etc.) 

2. Causal Attributions (MOT-CA) - statements that appear to be giving a cause for some previously mentioned event usually 
following some metacognitive statement (e.g., I don’t understand this because the book is so poorly written ) 

3. Confidence/Self-efficacv (MOT-SE) - statement reflects feeling success or confidence in any cognitive activity or 
behavior, such as comprehension, understanding, use of strategies, planning, etc. Hard to tell from monitoring, but efficacy 
statements should not be tracking oriented. 

4. General motivation (MOT) - anything that seems like motivation related such as references to effort or persistence, 
expectancies, etc. 
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Appendix C 

Examples of Segmented and Scored Protocols 

Subject #113 
During Reading Statements 
Pretest 

Cog. Processing Statement # Protocol Statement 



PK-PER 


S 1 


We were just talking about this today 


NS 


S 2 


(mumbled) 


NS 


S 3 


Didn't do all that (?) 


PK-REL 


S 4 


Yea, right. 


PK-REL 


S 5 


But they all failed 


PK-REL 


S 6 


Good luck 




(prompted to speak louder) 




MON 


S 7 


Oh, I get it (not in response to prompt) 


NS 


S 8 


He can't swear that one (?) 


PK-REL 


S 9 


Not true 


MON-ENVIRO S 10 




What time is it? 


DP-C 


S 11 


I wonder what the deal is with ( 


PK-PER 


S 12 


We already read about that. 


SP 


S 13 


Wyoming 



(long pause, page turned) 



S 14 Hum, ( {?}) ..(big sigh) 

S 15 Should be something about (?) 

(prompted to speak more loudly) 



PK-REL 


S 16 


PK-REL 


‘ S 17 


SP 


S 18 


NS 


S 19 


PK-PER 


S 20 


INT 


S 21 


PK-PER 


S 22 


NS 


S 23 




S 24 



Of course 

What do you think? 

Jim Crow 

A whole new way (??) 

I've heard of him. 

Damn, this is boring. 

Never heard of him. 
(?) 




I’m done. 
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Appendix C continued 



Subject #113 
During Reading Statements 
Posttest 



Cog. Processing 


Statement # 


Protocol Statement 


PK-REL 


SI 


Never heard of him 


NS 


S2 


We‘re not the army 


NS 


S3 


(?) 


MON 


S4 


What? 


NS . 


S5 


Why should I (?) 


MON-ENVIRO 


S6 


What? I wish they would (noise outside) 


DP-C 


S7 


Why commerce? 


DP-C 


S8 


Who are they? 


DP-C 


S9 


Could it be the other way around 


DP-S 


S10 


All these people 


DP-C 


Sll 


Which politician? 


DP-C 


S12 


What ‘Tea Pot'*/ 


PK-C 


S 13 


Isn’t that considered a bribe? 


M-EVAL 


S14 


I thought so. 


DP-C 


S15 


Why would he do that? 


DP-C 


S 16 


Who is Silent Cal? 


PK-PER 


S17 


MacAdoo (amused at unusual name) 


PK-PER 


S18 


Sure 


PK-REL 


S19 


Yea, right 


PK-REL 


S20 


Good 


PK-C 


S21 


How could you be both of these? 


NS 


S22 


Do I read this? (directed to experimenter) 


DP-C 


S23 


Why? (turned back on Progressives) 


PK-PER 


S24 


Sounds’s like Sim City. 


DP-C 


S25 


Why couldn’t they. 


NS 


S26 


Wanted their fill (1) 


PK-REL 


S27 


Didn’t they do that in Vietnam? 


DP-S 


S 28 


A lot of changing 


PK-REL 


S 29 


It wouldn’t work 
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Subject# 115 
During Reading Statements 
Pretest 



Cog. Processing 


Statement # 


Protocol Statement 


INT 


S 1 


So far this stuff is pretty interesting. I like it a lot. 


M-EVAL 


S 2 


I usually look up the definitions of the bold 
words while I’m reading to get a better idea of 
what its about, (does not do so now, though) 


M-EVAL 


S 3 

(long pause) 


I didn't really understand this one section part. 


INT 


S 4 


This part is sort of boring. / 


M-EVAL 




It doesn't really make sense. 


PK-REL 


S 5 


I think women should have had the right to vote 
all along, but I'm glad they got the right too. 


PK-PER 


S 6 


I think you should have to be like 15 yrs to vote, 
so people could , urn... 


MON 


S 7 


I lost my idea. 


INT 


S 8 


I don’t like to read the "Eye Witness to History" 
part'. I don’t like to read that kind of stuff. 


MON-ENVIRO 


S 9 


It's a little noisy right now. 


MON-ENVIRO 


S 10 


It's like really noisy 


PK-REL 


S 11 


I think the blacks should have been treated with 
the same respect. / 


PK-REL 




I think everybody is equal. 


MON-ENVIRO 


S 12 


It's major-league noise (laughing b/c kids singing 
Happy Birthday, way off key) 


PK-S 


S 13 


I think black should have been able to use the 
same restaurants and restrooms and streetcars as um, 
everybody else did. I don't think it should be 
seperate. 


MON-PER 


S 14 


I can't wait till lunch 


M-EVAL 


S 15 


I think this was an interesting section for the most 
part. Some parts were a little boring, but it was 
overall pretty interesting. 




S 16 


I’m done. 
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Appendix C continued 



Subject #115 

During Reading Statements 
Posttest 



Cog. Processing 


Statement # 


Protocol Statement 




(reading silently, turns 1st page) 


DP-C 


SI 


I’d like to know more about ’’keep Warren at home" 
(passage in text about Harding’s campaigning 
inabilities) 


DP-C 


S2 


I wonder why Harding called the League of 
Nations a fraud? 


MON-ENVIRO 


S3 


It’s noisy right now. 


DP 


S4 


I don't even see why Harding even ran for president. 
It says here he told someone it would be too hard for 
him. 


DP-S 


S5 


It said here that some of Harding's appointees screwed 
up their office, and he was the one who appointed 






’em. 


DP-C 


S6 


I want to know what he means when he said, 


SP 




"I can take care of my enemies. It's my friends that 
keep me walking the floors at night." 

I want to know what that means. 


DP-S 


SI 


I think Coolidge made a better president than Harding. 


DP 


S8 


I didn't think Harding was a fair man either. I 
think he was a crook 




(see text, re. Harding’s eulogy . Subject's opinion isn’t prior knowledge b/c 
probably didn't know about Harding till he read this passage - but he does know 




what a CROOK is) 




PK-PER 


S9 


That’s a cool saying (laughs) "Keep Cool with 
Coolidge" 




(turns page soon afterward) 






S10 


I’m through. 
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Description of Trainin g Procedures 

StudentS in my S0Cial studies class took in a reading-strategy training intervention during the early Spring semester, 
1995. All students in the class were required to participate in the strategy instruction as part of the regular classroom curriculum, 
although not all agreed to be participants in the study that accompanied this intervention. From a class of 24 students, 14 (58%) 
agreed to serve as subjects in the study. The reason most often cited for not participating in the study was apprehension about being 
videotaped during the think-aloud protocol. For adolescents who are self-conscious about their physical appearance, this was 
unacceptable even after repeated assurances that the tapes would only be used for purposes of the research. On the whole, 
students who did not agree to participate in the study tended to by shy, soft-spoken, and female. Reading ability did not seem to be a 
factor in determining participation in the study. 

Who l e-c l ass i nstruct i on on the first day . On the first day of instruction, I delivered whole-class and group instruction to 
all students (participants and non-participants). There were three components to my whole-class instruction. First, I described the 
procedures for the reading groups (e.g., how to group desks, where assignments for independent work would be posted, what would 
be happening in the reading group, what should be done when students were not meeting with their group, where and when quiz 
grades would be posted). Second, after reading a paragraph from the social studies textbook, 1 modeled how to use the four 
reading strategies we would be using in the reading groups. I repeated the reading, describing, and modeling sequence two times. 
Third, I described reasons why it was important that students learn and become effective at using these strategies. 

GlQ UP-instruction on the first and subsequent davs . Following this whole-class instruction, 1 gave students their non- 
group (independent) assignments for social studies and the First reading group (D group) was convened. On subsequent days there 
was no whole-class instruction. 

V/hile I met with each group, all other students were expected to complete their independent assignments at their own 
desks. On the first day of group reading (and each day during the first week), I reviewed, with each group, the four reading 
strategies and modeled how to use the strategies. After the first week, I usually omitted the review of strategies and simply read 
and modeled how to use the reading strategies. After I had modeled the use of the reading strategies, we then read from our social 
studies textbook, proceeding one paragraph at a time. After all members had finished reading the first paragraph, 1 chose one 
member to (a) identify the main idea of the paragraph by summarizing the paragraph in a sentence, (b) ask a main-idea question by 
transforming the main-idea statement into a question, (c) clarify any part of the paragraph that was not clear, and (d) to predict 
what the author would most likely write about in subsequent paragraphs. When each reader had completed these steps, the next 
paragraph was read and I chose another group leader. I encouraged all members of the group to read and go through each step 
even if it was not their turn to lead the group so that they could offer feedback to the other group members. When each group 
member had been given an opportunity to lead their group, students were asked to return to their normal seats and begin (or 
resume) their independent assignment. All of these steps were repeated for each reading group during each day of the training 

Within the groups, I tried to facilitate inter-student feedback regarding main ideas construction and clarifications. 1 
encouraged group members to offer suggestions to the leader or to ask for points of clarification. 1 also reminded students of my 
own goal to turning over direction to the group members and to minimize my part in directing each individual student. For reasons 
that are unclear, however, students rarely participated beyond fulfilling their requirement of using the strategies when they were 
called upon to do so. Had I turned responsibility for reading over to the group members, perhaps they would have become more 
interactive and independent. 

Since I met with only one group at a time, students who were not reading with their groups were required to complete 
other social studies assignment (usually section review question). These assignment were concerned with a portion of the chapter 
that had been read on the previous night for homework or, as the training progressed, the independent assignment corresponded to 
the reading that had been done in groups on the previous day. When students completed the individual assignment, they were free 
to complete other homework assignments or to listen to the responses of the reading groups that was meeting. Students were no 
allowed to talk or move about the room during this time. 

After meeting with each group (Groups D, V, N, T), I administered a short quiz to the whole class. I administered 
reading quizzes three days per week during training. I did not quiz students daily primarily because of time constraints related to 
their daily schedule. There was insufficient time to meet with each group and provide time for a quiz during the allotted social 
studies time. , 



Each quiz covered a section of the chapter that had not yet been read. The quizzes consisted of at least five open-ended 
questions. I constructed questions for the quizzes by first reading and stating the main idea of a paragraph, and then transforming 
this statement into a main idea question. I attempted to encourage students to use the reading strategies in this way by constructing 
the questions according to the same method that students were being taught to use in their reading groups. 

At the end of each week of training, I asked students to complete a training manipulation check. The manipulation 
check consisted of a passage from the textbook (one that had not previously been read in groups), and a brief questionnaire that 
required students to record the results of their strategy use in writing. For example, after reading the passage, students were asked 
to wnte the main idea of the first paragraph in the passage, write a main-idea question, describe anything that they did not 
understand, and to predict what they thought might come next after the end of the passage. All passages were taken from the 
regular social studies text, and all students read directly from their own textbook. 

In terms of the interactive dynamics of each group, each one seemed to function in qualitatively different manners. 

Group D was extremely reserved, slow to speak when asked questions, usually responded to each strategy question by rereading 
the passage and then remaining silent. On the whole, they appeared unmotivated due to a lack of confidence in their own abilities. 
One member of the group, however, was an exceptionally good reader, and he displayed signs of frustration with other members 
who were less able. He did not attempt to help other members accomplish these tasks. Three of the 6 members in this group were 
participants in the study. 6 v 
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Group V, on the other hand, was eager (relative to Group D) to participate in the group reading activities. Their 
responses were typically thoughtful and of a higher caliber. As a group, we frequently digressed into discussions about the content 
ot the passage. Members often offered alternative main ideas or suggested ways that one member might enhance a question or 
main-idea statement. All members of this group were of similar ability, though not all were equally confident in their abilities. Four 
or the 6 members in this group were participants in the study. 

i . Group N also could be described as quiet, but competent in their ability to perform the required procedure. This group 
could be characterized, however, as unmotivated to read about social studies or to learn new reading strategies. It was one of the 
O _ in this group that, on two separate occasions, asked about the purpose behind the strategy training. This group typically 

did not interact with each other during reading groups or in more social settings outside of the classroom. All 6 of the members in 
this group were participants in the study. ■ ^ 
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Group T was by far the group with the highest average ability for reading. This group functioned effectively and 
efficiently together with little need to offer corrective feedback. They easily mastered the four-step procedure used during training 
and were able to perform derivations (e.g., reading more than one paragraph before summarizing; performing the four steps, but 
also linking their passage to what had been stated before) of the steps with little difficulty. This group could also be characterized 
as extremely shy. Of the 6 members in this group, only 1 member agreed to participate in the study. 

Evaluation of the Training Procedure 

The goal of the present study was to examine the effects of strategy training on readers’ real-time cognitive, 
metacognitive, and motivational processes. I chose to deliver the strategy training using Palincsar and Brown’s (1984) reciprocal 
teaching method. 

In terms of the quality of instructional delivery, I believe there was a fairly high degree of faithfulness to the method as 
described by Palincsar and Brown (1984). There was, however, one important exception related to the fading of teacher support. 
Palincsar and Brown described their subjects as capable and willing to assume leadership roles in the group settings. My students, 
however, were willing to interact with me, but very reluctant to interact with other group members (Group V was and exception to 
this rule). Therefore, I was unwilling to risk the “fumbling” that Pressley and McCormick (1995, p. 243) described. In this sense, 
my inability to motivate students to participate in the group setting represents a failure with regard to my instructional delivery. 

In terms of the effectiveness of this intervention for enhancing readers’ real-time engagement in cognitive and 
metacognitive activity, it appears to me that the study was a practical failure. The changes that did occur were encouraging, and 
other factors may have contributed to a lack of significant results (e.g., small sample size). Based solely on the evidence from the 
analyses of the think-aloud protocols, however, there appears to have been no practical effect on readers’ strategy use. 

This conclusion makes understanding readers’ changes in comprehension performance somewhat puzzling. It also 
raises the question proposed by previous researchers (e.g., Bereiter & Bird, 1985; Wade et al., 1993) concerning the real-time 
relationship between strategy use and comprehension performance. 
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